Refactor(bootstrapper): add force_prebuilt flag, remove _handle_test_mode_failure#894
Conversation
17aef1b to
f31d17c
Compare
src/fromager/bootstrapper.py
Outdated
| build_error, | ||
| ) | ||
| try: | ||
| wheel_url, fallback_version = self._resolve_prebuilt_with_history( |
There was a problem hiding this comment.
We shouldn't need to resolve again, we have the version we want, we should just be able to call _bootstrap_impl again with the force flag set to true.
src/fromager/bootstrapper.py
Outdated
| fallback_error, | ||
| exc_info=True, | ||
| ) | ||
| # Record the original build error, not the fallback error |
There was a problem hiding this comment.
We need to record the original error even if the fallback is successful. We could also record the fallback error, as additional information, because it would be useful to know that we weren't able to do anything at all for the package.
There was a problem hiding this comment.
Have changed the code as per your suggestion.
3ffb492 to
3d7eb22
Compare
…prebuilt flag Move test-mode prebuilt fallback from _build_from_source into bootstrap(). On build failure, bootstrap() now records the error and retries via _bootstrap_impl(force_prebuilt=True), which resolves and downloads a pre-built wheel internally. This removes _handle_test_mode_failure and the req_type parameter from _build_from_source. Closes: python-wheel-build#892 Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Lalatendu Mohanty <lmohanty@redhat.com>
3d7eb22 to
6663428
Compare
| except Exception as build_error: | ||
| if not self.test_mode: | ||
| raise | ||
|
|
There was a problem hiding this comment.
You could add a check here to avoid trying again if the package is already marked as pre-built. I don't know how often that would come up, but in a deep dependency graph it would short-cut an expensive and useless operation.
| @@ -615,9 +683,12 @@ def _handle_build_requirements( | |||
|
|
|||
| for dep in self._sort_requirements(build_dependencies): | |||
There was a problem hiding this comment.
We have several loops doing this same thing now. Maybe a future PR could refactor those into a method to bootstrap an iterable of requirements. Don't update this one with that change, though.
|
@coderabbitai review |
|
@coderabbitai help |
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
📝 WalkthroughWalkthroughbootstrap() now records build exceptions in test mode, then retries using a pre-built wheel via Changes
Sequence Diagram(s)sequenceDiagram
participant Client as "bootstrap()"
participant Impl as "_bootstrap_impl()"
participant Builder as "Build system"
participant Resolver as "Prebuilt resolver"
participant DL as "Downloader"
participant Logger as "Logger"
Client->>Impl: request bootstrap(req)
Impl->>Builder: attempt build from source
alt build succeeds
Builder-->>Impl: build artifact + version
Impl-->>Client: return Version
Client->>Logger: log installed version
else build raises
Builder-->>Impl: raises build_error
Impl->>Logger: record build_error (test mode)
Impl->>Resolver: resolve prebuilt wheel (force_prebuilt)
alt prebuilt resolved
Resolver-->>Impl: wheel_url + fallback_version
Impl->>DL: download wheel_url
DL-->>Impl: wheel file
Impl-->>Client: return fallback Version
Client->>Logger: log prebuilt usage (warn on mismatch)
else prebuilt fails
Resolver-->>Impl: raises fallback_error
Impl->>Logger: log fallback_error with exc_info
Impl-->>Client: re-raise build_error chained to fallback_error
end
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Poem
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
@coderabbitai full review |
✅ Actions performedFull review triggered. |
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@src/fromager/bootstrapper.py`:
- Around line 494-500: The try/except that calls self.bootstrap(req=dep,
req_type=RequirementType.INSTALL) should only swallow exceptions in test_mode if
the failure has been recorded to failed_packages; update the except block to
verify the exception was logged (e.g., check failed_packages or that
_record_test_mode_failure was called) and re-raise if not, to avoid hiding
failures from finalize(); also apply the same guard to the analogous top-level
loop handling in the bootstrap command (ensure failures like those raised in
_add_to_graph / ctx.write_to_graph_to_file are recorded before suppressing).
- Around line 393-404: The code updates resolved_version to fallback_version
after bootstrap() has already called bookkeeping functions (_add_to_graph,
_mark_as_seen, _track_why), which can attach children to the old version and
corrupt build-order/deduping; move the prebuilt resolution logic so that any
fallback_version check and assignment of resolved_version (and source_url =
wheel_url) happens before bootstrap()/before calls to _add_to_graph(),
_mark_as_seen(), and _track_why(), or alternatively raise an explicit error on
mismatch; specifically update the flow around
resolved_version/fallback_version/source_url/wheel_url so resolved_version is
final prior to invoking the bookkeeping helpers referenced above.
- Around line 383-392: The current call to self._resolver.resolve in the
pbi.pre_built / force_prebuilt branch uses self.why[-1][1] as parent_req but
_track_why() has already pushed the current req, so that yields the wrong
parent; instead determine the real parent before/independent of the push and
pass it to resolve (e.g. use the previous entry self.why[-2][1] when self.why
has at least two entries, otherwise None) so that
self._resolver.resolve(req=..., req_type=req_type, parent_req=actual_parent,
pre_built=True) receives the correct parent package.
In `@tests/test_bootstrap_test_mode.py`:
- Around line 299-327: The test test_fallback_succeeds_records_build_failure
fails to capture the INFO log "successfully used pre-built wheel"; set caplog to
capture INFO level before invoking bt.bootstrap by calling
caplog.set_level(logging.INFO) (ensure logging is imported) so the assertion on
caplog.text reliably sees the logger.info message emitted by
src/fromager/bootstrapper.py; preserve existing mocks and assertions for
bt.failed_packages and exception_message.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: cc4d34b8-ab68-4cd0-89bd-02b3457eaa22
📒 Files selected for processing (4)
src/fromager/bootstrapper.pysrc/fromager/commands/bootstrap.pytests/test_bootstrap_test_mode.pytests/test_bootstrapper.py
| if pbi.pre_built or force_prebuilt: | ||
| if force_prebuilt: | ||
| # Resolve prebuilt wheel URL (source_url is the sdist URL) | ||
| parent_req = self.why[-1][1] if self.why else None | ||
| wheel_url, fallback_version = self._resolver.resolve( | ||
| req=req, | ||
| req_type=req_type, | ||
| parent_req=parent_req, | ||
| pre_built=True, | ||
| ) |
There was a problem hiding this comment.
Pass the actual parent into the prebuilt resolver.
By the time this branch runs, _track_why() has already pushed req, so self.why[-1][1] is the package being retried, not its parent. Top-level fallbacks therefore resolve with parent_req=req, and transitive fallbacks resolve with the wrong parent.
🔧 Minimal fix
- parent_req = self.why[-1][1] if self.why else None
+ parent_req = self.why[-2][1] if len(self.why) > 1 else None🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/fromager/bootstrapper.py` around lines 383 - 392, The current call to
self._resolver.resolve in the pbi.pre_built / force_prebuilt branch uses
self.why[-1][1] as parent_req but _track_why() has already pushed the current
req, so that yields the wrong parent; instead determine the real parent
before/independent of the push and pass it to resolve (e.g. use the previous
entry self.why[-2][1] when self.why has at least two entries, otherwise None) so
that self._resolver.resolve(req=..., req_type=req_type,
parent_req=actual_parent, pre_built=True) receives the correct parent package.
| try: | ||
| self.bootstrap(req=dep, req_type=RequirementType.INSTALL) | ||
| except Exception: | ||
| if not self.test_mode: | ||
| raise | ||
| # Failure already recorded in failed_packages; continue. | ||
| self.progressbar.update() |
There was a problem hiding this comment.
Only suppress child bootstrap errors after confirming they were recorded.
These blocks assume every test-mode exception from bootstrap() has already appended to failed_packages, but that is not guaranteed for failures before _record_test_mode_failure() runs, such as self.ctx.write_to_graph_to_file() inside _add_to_graph() for transitive dependencies. Swallowing those exceptions here can let finalize() report success even though dependencies were skipped. The same guard is needed in the top-level loop in src/fromager/commands/bootstrap.py.
🔧 Suggested guard
+ failures_before = len(self.failed_packages)
try:
self.bootstrap(req=dep, req_type=RequirementType.INSTALL)
except Exception:
- if not self.test_mode:
+ if (
+ not self.test_mode
+ or len(self.failed_packages) == failures_before
+ ):
raise
# Failure already recorded in failed_packages; continue.Also applies to: 686-692
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/fromager/bootstrapper.py` around lines 494 - 500, The try/except that
calls self.bootstrap(req=dep, req_type=RequirementType.INSTALL) should only
swallow exceptions in test_mode if the failure has been recorded to
failed_packages; update the except block to verify the exception was logged
(e.g., check failed_packages or that _record_test_mode_failure was called) and
re-raise if not, to avoid hiding failures from finalize(); also apply the same
guard to the analogous top-level loop handling in the bootstrap command (ensure
failures like those raised in _add_to_graph / ctx.write_to_graph_to_file are
recorded before suppressing).
| def test_fallback_succeeds_records_build_failure( | ||
| self, tmp_context: context.WorkContext, caplog: pytest.LogCaptureFixture | ||
| ) -> None: | ||
| """Test that build failure is always recorded, even when fallback succeeds.""" | ||
| bt = bootstrapper.Bootstrapper(ctx=tmp_context, test_mode=True) | ||
| req = Requirement("test-package>=1.0") | ||
|
|
||
| with ( | ||
| mock.patch.object( | ||
| bt, | ||
| "resolve_version", | ||
| return_value=("https://sdist.url", Version("1.0")), | ||
| ), | ||
| mock.patch.object(bt, "_add_to_graph"), | ||
| mock.patch.object(bt, "_has_been_seen", return_value=False), | ||
| mock.patch.object(bt, "_mark_as_seen"), | ||
| # First call fails (build), second succeeds (fallback with force_prebuilt) | ||
| mock.patch.object( | ||
| bt, | ||
| "_bootstrap_impl", | ||
| side_effect=[RuntimeError("Build failed"), None], | ||
| ), | ||
| ): | ||
| bt.bootstrap(req=req, req_type=RequirementType.TOP_LEVEL) | ||
|
|
||
| # Build failure is always recorded | ||
| assert len(bt.failed_packages) == 1 | ||
| assert "Build failed" in bt.failed_packages[0]["exception_message"] | ||
| assert "successfully used pre-built wheel" in caplog.text |
There was a problem hiding this comment.
Raise caplog to INFO before asserting the fallback success message.
"successfully used pre-built wheel" is logged with logger.info() in src/fromager/bootstrapper.py, so this assertion is fragile unless the test lowers the capture threshold first.
🔧 Suggested fix
def test_fallback_succeeds_records_build_failure(
self, tmp_context: context.WorkContext, caplog: pytest.LogCaptureFixture
) -> None:
+ caplog.set_level("INFO", logger="fromager.bootstrapper")
bt = bootstrapper.Bootstrapper(ctx=tmp_context, test_mode=True)
req = Requirement("test-package>=1.0")🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@tests/test_bootstrap_test_mode.py` around lines 299 - 327, The test
test_fallback_succeeds_records_build_failure fails to capture the INFO log
"successfully used pre-built wheel"; set caplog to capture INFO level before
invoking bt.bootstrap by calling caplog.set_level(logging.INFO) (ensure logging
is imported) so the assertion on caplog.text reliably sees the logger.info
message emitted by src/fromager/bootstrapper.py; preserve existing mocks and
assertions for bt.failed_packages and exception_message.
|
@coderabbitai configuration |
|
Configuration used: defaults CodeRabbit configurationlanguage: en-US
tone_instructions: ''
early_access: false
enable_free_tier: true
inheritance: false
reviews:
profile: chill
request_changes_workflow: false
high_level_summary: true
high_level_summary_instructions: ''
high_level_summary_placeholder: '@coderabbitai summary'
high_level_summary_in_walkthrough: false
auto_title_placeholder: '@coderabbitai'
auto_title_instructions: ''
review_status: true
review_details: false
commit_status: true
fail_commit_status: false
collapse_walkthrough: true
changed_files_summary: true
sequence_diagrams: true
estimate_code_review_effort: true
assess_linked_issues: true
related_issues: true
related_prs: true
suggested_labels: true
labeling_instructions: []
auto_apply_labels: false
suggested_reviewers: true
auto_assign_reviewers: false
in_progress_fortune: true
poem: true
enable_prompt_for_ai_agents: true
path_filters: []
path_instructions: []
abort_on_close: true
disable_cache: false
slop_detection:
enabled: true
auto_review:
enabled: true
description_keyword: ''
auto_incremental_review: true
auto_pause_after_reviewed_commits: 5
ignore_title_keywords: []
labels: []
drafts: false
base_branches: []
ignore_usernames: []
finishing_touches:
docstrings:
enabled: true
unit_tests:
enabled: true
simplify:
enabled: false
custom: []
pre_merge_checks:
override_requested_reviewers_only: false
docstrings:
mode: warning
threshold: 80
title:
mode: warning
requirements: ''
description:
mode: warning
issue_assessment:
mode: warning
custom_checks: []
tools:
ast-grep:
rule_dirs: []
util_dirs: []
essential_rules: true
packages: []
shellcheck:
enabled: true
ruff:
enabled: true
markdownlint:
enabled: true
github-checks:
enabled: true
timeout_ms: 90000
languagetool:
enabled: true
enabled_rules: []
disabled_rules: []
enabled_categories: []
disabled_categories: []
enabled_only: false
level: default
biome:
enabled: true
hadolint:
enabled: true
swiftlint:
enabled: true
phpstan:
enabled: true
level: default
phpmd:
enabled: true
phpcs:
enabled: true
golangci-lint:
enabled: true
yamllint:
enabled: true
gitleaks:
enabled: true
trufflehog:
enabled: true
checkov:
enabled: true
tflint:
enabled: true
detekt:
enabled: true
eslint:
enabled: true
flake8:
enabled: true
fortitudeLint:
enabled: true
rubocop:
enabled: true
buf:
enabled: true
regal:
enabled: true
actionlint:
enabled: true
pmd:
enabled: true
clang:
enabled: true
cppcheck:
enabled: true
opengrep:
enabled: true
semgrep:
enabled: true
circleci:
enabled: true
clippy:
enabled: true
sqlfluff:
enabled: true
trivy:
enabled: true
prismaLint:
enabled: true
pylint:
enabled: true
oxc:
enabled: true
shopifyThemeCheck:
enabled: true
luacheck:
enabled: true
brakeman:
enabled: true
dotenvLint:
enabled: true
htmlhint:
enabled: true
stylelint:
enabled: true
checkmake:
enabled: true
osvScanner:
enabled: true
blinter:
enabled: true
smartyLint:
enabled: true
emberTemplateLint:
enabled: true
psscriptanalyzer:
enabled: true
chat:
art: true
allow_non_org_members: true
auto_reply: true
integrations:
jira:
usage: auto
linear:
usage: auto
knowledge_base:
opt_out: false
web_search:
enabled: true
code_guidelines:
enabled: true
filePatterns: []
learnings:
scope: auto
issues:
scope: auto
jira:
usage: auto
project_keys: []
linear:
usage: auto
team_keys: []
pull_requests:
scope: auto
mcp:
usage: auto
disabled_servers: []
linked_repositories: []
code_generation:
docstrings:
language: en-US
path_instructions: []
unit_tests:
path_instructions: []
issue_enrichment:
auto_enrich:
enabled: false
planning:
enabled: true
auto_planning:
enabled: true
labels: []
labeling:
labeling_instructions: []
auto_apply_labels: false
|
There was a problem hiding this comment.
♻️ Duplicate comments (1)
src/fromager/bootstrapper.py (1)
383-392:⚠️ Potential issue | 🟠 MajorFix
parent_reqto reference the actual parent, not the current package.At line 386,
self.why[-1][1]retrieves the current package being bootstrapped (which was pushed onto the stack by_track_why), not its parent. This causes two problems:
- For top-level packages:
self.whycontains only the current package, soparent_reqis incorrectly set to the package itself rather thanNone.- For transitive dependencies:
parent_reqpoints to the current package instead of the actual parent.The correct fix uses
self.why[-2]to access the parent:Suggested fix
if force_prebuilt: # Resolve prebuilt wheel URL (source_url is the sdist URL) - parent_req = self.why[-1][1] if self.why else None + parent_req = self.why[-2][1] if len(self.why) > 1 else None wheel_url, fallback_version = self._resolver.resolve(🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/fromager/bootstrapper.py` around lines 383 - 392, The code sets parent_req using self.why[-1][1], which refers to the current package pushed by _track_why rather than its parent; update the logic inside the pbi.pre_built/force_prebuilt branch so parent_req is None when self.why has only one entry and otherwise uses the previous frame's request (i.e. self.why[-2][1]); ensure this change is applied where parent_req is passed into self._resolver.resolve(req=..., req_type=..., parent_req=parent_req, pre_built=True).
🧹 Nitpick comments (2)
src/fromager/bootstrapper.py (1)
494-500: Consider adding a guard to verify failure was recorded before suppressing.This exception handling assumes that any exception raised by
bootstrap()has already been recorded infailed_packages. However, if an exception occurs early inbootstrap()(e.g., in_add_to_graph()at line 272, which is before the maintryblock), it would be swallowed without being recorded.The same concern applies to the identical pattern at lines 686-692.
🛡️ Optional guard pattern
for dep in self._sort_requirements(install_dependencies): with req_ctxvar_context(dep): + failures_before = len(self.failed_packages) try: self.bootstrap(req=dep, req_type=RequirementType.INSTALL) except Exception: - if not self.test_mode: + if not self.test_mode or len(self.failed_packages) == failures_before: raise # Failure already recorded in failed_packages; continue. self.progressbar.update()🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/fromager/bootstrapper.py` around lines 494 - 500, The exception handler in bootstrapper.bootstrap that currently swallows all exceptions when self.test_mode is true should first verify the failure was recorded in self.failed_packages (or whatever structure records failures) before suppressing; change the except Exception: blocks (the one wrapping self.bootstrap(req=dep, req_type=RequirementType.INSTALL) and the identical block at the later lines) to except Exception as e: and if self.test_mode and dep in self.failed_packages: continue else: raise (this ensures exceptions thrown early in bootstrap (e.g., in _add_to_graph) are not silently swallowed).src/fromager/commands/bootstrap.py (1)
185-192: Pattern aligns with_bootstrap_impl()dependency loops.The exception-handling logic here mirrors the approach used in
_bootstrap_impl()for install and build dependencies. The assumption that failures are "already recorded inbt.failed_packages" holds for resolution and build errors caught withinbootstrap().However, note that a past review flagged a subtle edge case: if an exception occurs in
_add_to_graph()(before the maintryblock inbootstrap()), it would be swallowed here without being recorded. Consider adding a guard to verify the failure was actually recorded before suppressing, as suggested in prior reviews.🛡️ Optional guard to verify failure was recorded
for req in resolved_reqs: token = requirement_ctxvar.set(req) + failures_before = len(bt.failed_packages) try: bt.bootstrap(req, requirements_file.RequirementType.TOP_LEVEL) except Exception: - if not test_mode: + if not test_mode or len(bt.failed_packages) == failures_before: raise # In test mode, failure is already recorded in # bt.failed_packages; continue to next requirement. progressbar.update() requirement_ctxvar.reset(token)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/fromager/commands/bootstrap.py` around lines 185 - 192, The current except block after calling bt.bootstrap(req, ...) can swallow exceptions that weren't recorded (e.g., those raised in _add_to_graph()); update the handler to capture the exception (except Exception as e:) and, when test_mode is True, verify the failure was actually recorded in bt.failed_packages for the current requirement (use the same key/identifier bt uses for failures) before suppressing; if the requirement is not present in bt.failed_packages, re-raise the exception (or propagate e) so unexpected errors aren’t silently dropped. Ensure this logic is applied around the bt.bootstrap(req, ...) call and references bt.failed_packages and the current req identifier.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Duplicate comments:
In `@src/fromager/bootstrapper.py`:
- Around line 383-392: The code sets parent_req using self.why[-1][1], which
refers to the current package pushed by _track_why rather than its parent;
update the logic inside the pbi.pre_built/force_prebuilt branch so parent_req is
None when self.why has only one entry and otherwise uses the previous frame's
request (i.e. self.why[-2][1]); ensure this change is applied where parent_req
is passed into self._resolver.resolve(req=..., req_type=...,
parent_req=parent_req, pre_built=True).
---
Nitpick comments:
In `@src/fromager/bootstrapper.py`:
- Around line 494-500: The exception handler in bootstrapper.bootstrap that
currently swallows all exceptions when self.test_mode is true should first
verify the failure was recorded in self.failed_packages (or whatever structure
records failures) before suppressing; change the except Exception: blocks (the
one wrapping self.bootstrap(req=dep, req_type=RequirementType.INSTALL) and the
identical block at the later lines) to except Exception as e: and if
self.test_mode and dep in self.failed_packages: continue else: raise (this
ensures exceptions thrown early in bootstrap (e.g., in _add_to_graph) are not
silently swallowed).
In `@src/fromager/commands/bootstrap.py`:
- Around line 185-192: The current except block after calling bt.bootstrap(req,
...) can swallow exceptions that weren't recorded (e.g., those raised in
_add_to_graph()); update the handler to capture the exception (except Exception
as e:) and, when test_mode is True, verify the failure was actually recorded in
bt.failed_packages for the current requirement (use the same key/identifier bt
uses for failures) before suppressing; if the requirement is not present in
bt.failed_packages, re-raise the exception (or propagate e) so unexpected errors
aren’t silently dropped. Ensure this logic is applied around the
bt.bootstrap(req, ...) call and references bt.failed_packages and the current
req identifier.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 85cc2287-b3f6-4855-9d24-7ba70227972c
📒 Files selected for processing (4)
src/fromager/bootstrapper.pysrc/fromager/commands/bootstrap.pytests/test_bootstrap_test_mode.pytests/test_bootstrapper.py
Move test-mode prebuilt fallback from _build_from_source into bootstrap().
On build failure, bootstrap() now records the error and retries via
_bootstrap_impl(force_prebuilt=True), which resolves and downloads a
pre-built wheel internally. This removes _handle_test_mode_failure and
the req_type parameter from _build_from_source.
Also added tests for prebuilt fallback path in bootstrap()
Closes #892 , Depends on #893
Summary by CodeRabbit
New Features
Bug Fixes
Tests